Interlacing has been used since the early days of television as a means of conserving the bandwidth required to transmit a video signal. The NTSC, PAL and SECAM television systems all make use of interlacing whereby odd and even lines comprising a video image are transmitted in separate fields. Even with the advent of high definition television and digital transmission systems, interlacing is still commonly used in order to conserve bandwidth. For instance, 1920×1080i is a commonly used format containing 1080 interlaced lines, each line consisting of 1920 pixels.
Display devices based on cathode ray tube (CRT) technology can display interlaced video without the need for prior conversion since the electron beam can be made to scan the screen in an interlaced fashion which matches the interlacing of the received video signal. Flat panel display devices including LCD, plasma, OLED and other technologies are usually progressively scanned whereby odd and even lines comprising a video image are displayed together. Therefore, an interlaced video source must first be converted to progressive format before being displayed on these types of devices. Furthermore, conversion to progressive format when performed effectively, reduces or eliminates certain artifacts associated with interlacing including line flicker.
Numerous methods have been proposed for converting a video signal from interlaced to progressive format. For example, intra-field interpolation, sometimes referred to as “bob”, may be used to generate the missing pixels between the lines of an interlaced field in order to create a complete video frame. In this case, missing pixels are generated by interpolating between the values of the vertical neighbors just above and just below the missing pixel. This simple method is frequently used in low cost display devices such as computer monitors since it does not require the use of DRAM for storing entire fields of video. The method performs reasonably well on moving image portions but poorly on static portions due to loss of vertical resolution. Inter-field interpolation, sometimes referred to as “weave”, is another commonly understood method. In this case, the missing pixels between the lines of an interlaced field are generated by interpolating between the values of the temporal neighbors or by simply copying the value of the temporal neighbor just before or just after the missing pixel (zero order interpolation). This method produces an ideal output for static image portions, but results in severe motion artifacts for moving portions due to the merging of video fields sampled at different points in time. Consequently, this method is rarely used except in special circumstances, such as video derived from film with a known cadence.
Motion adaptive format conversion attempts to exploit the best of both of the previously described methods. First, the level of motion in the vicinity of a missing pixel is measured according to various methods which are known to those skilled in the art. It is beyond the scope of this disclosure to describe all the possible methods of motion detection, however, most involve calculating the differences between the values of pixels at corresponding horizontal and vertical positions in successive video fields of the same or opposite parity. In the case where the level of motion is low or zero, indicative of a static image portion, the missing pixel is generated using inter-field interpolation so as to maximize vertical resolution. In the case where the level of motion is high, indicative of a moving image portion, the missing pixel is generated using intra-field interpolation so as to avoid motion artifacts, albeit at the expense of vertical resolution. Practical realizations of this method typically involve a “soft switch” between the static and motion cases based on the level of motion to help reduce the appearance of scintillation artifacts that may occur at or near the motion threshold.
Despite the use of soft switching as described above, the appearance of scintillation artifacts due to the presence of noise and at the onset of motion remains a major limitation of motion adaptive techniques. Various solutions have been proposed to address this issue including those described in U.S. Pat. No. 6,784,942 (Selby). In one embodiment of the above, vertical-temporal interpolation is used in place of intra-field interpolation corresponding to the case where the level of motion detected is high. The method of vertical-temporal interpolation preserves vertical detail in comparison with purely vertical interpolation by deriving an enhancing high vertical frequency contribution from one or more adjacent fields in addition to the contribution from the current field for which the missing pixel is sought. Furthermore, since the coefficients for the adjacent field contributions sum substantially to zero, severe motion artifacts are generally avoided, particularly for horizontal motion against a flat background. However, artifacts can be produced for vertical motion greater than a certain velocity at which point the high frequency contribution from the adjacent field(s) becomes detrimental rather than enhancing. In addition, even for horizontal motion, artifacts can be produced near the corners of objects or anywhere that a vertical detail moves to overlap a relatively flat area. In a second embodiment of the above reference, the detrimental effect of the high frequency contribution at higher velocities is mitigated by performing purely vertical interpolation at high levels of motion, vertical-temporal interpolation at intermediate levels of motion and purely temporal interpolation in the static case and at very low levels of motion. The problem with this approach is that owing to the method by which motion is usually detected, even motion at a low velocity which might otherwise benefit from the use of vertical-temporal interpolation, may register as a high level of motion instead. This is because the method by which motion is usually detected, which involves looking at the differences between the values of pixels in successive video fields, gives a measure of motion which is not necessarily proportional to actual velocity. Consequently, full transition to purely vertical interpolation associated with the high motion case often occurs for low velocities which defeats the benefit of vertical-temporal interpolation and fails to solve the problem of scintillation artifacts which may occur at or near the motion threshold.
It is an objective of the present invention to provide a method of motion adaptive deinterlacing in which the problem of scintillation artifacts in the presence of low motion is adequately addressed while minimizing artifacts in the high motion case.